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Abstract. We prove results about the intersection of the p-rank strata and the boundary of the 
moduli space of hyperelliptic curves in characteristic p > 3. This yields a strong technique that al- 
lows us to analyze the stratum fig of hyperelliptic curves of genus g and p-rank /. Using this, we 
prove that the endomorphism ring of the Jacobian of a generic hyperelliptic curve of genus g and 
p-rank / is isomorphic to Z if ^ > 4. Furthermore, we prove that the Z / I'-monodromy of every irre- 
ducible component of fig is the symplectic group Sp2^(Z/£) if §■ > 4 or / > 1, and £ ^ p is an odd 
prime (with mild hypotheses on £ when / = 0). These results yield numerous applications about 
the generic behavior of hyperelliptic curves of given genus and p-rank over finite fields, including 
applications about Newton polygons, absolutely simple Jacobians, class groups and zeta fimctions. 



1. Introduction 

Suppose C is a smooth connected projective hyperelliptic curve of genus g > I over an alge- 
braically closed field k of characteristic p > 3. The Jacobian Pic^(C) is a principally polarized 
abelian variety of dimension g. The number of physical p-torsion points of Pic'^(C) is pf for some 
integer /, called the p-rank of C, with < f < g. 

Let Hg be the moduli space over k of smooth connected projective hyperelliptic curves of genus 

g; it is a smooth Deligne-Mumford stack over k. The p-rank induces a stratification Hg = U'Hg by 

locally closed reduced substacks Hg, whose geometric points correspond to hyperelliptic curves 
of genus g and p-rank /. 

In this paper, we prove three cumulative results about Hg. The first is about the boundary 

of Hg) specifically, when g > 2, we prove that the boundary of every irreducible component 

of Hg contains the moduli point of some singular curve which is a tree of elliptic curves and 
which has p-rank /. The second is that the Jacobian of a generic curve of genus g and p-rank / 
has endomorphism ring Z if g > 4. The third is that, for an odd prime number £ distinct from 

p, the mod-i monodromy group of every irreducible component of Hg is the symplectic group 
Sp2^(Z/£) if ^ > 4 or / > 1 (with mild hypotheses on £ when / = 0). Heuristically, this means 
that p-rank constraints alone do not force the existence of extra automorphisms (or other algebraic 
cycles) on a family of hyperelliptic curves. 

We now state the results of this paper more precisely. 

Theorem |3.lH c). Suppose p is an odd prime, g > 2, and < f < g. Let S be an irreducible component 

ofHg, the p-rank f stratum in Hg. Then the closure S of S in Hg contains the moduli point of some tree of 
g elliptic curves, of which f are ordinary and g — f are supersingular. 

Our proof does not yield much information on the structure of the tree in Theorem B.llt how- 
ever, once the tree's structure is fixed, we prove that any choice of labeling of / components as 
ordinary and g — f components as supersingular will occur for some moduli point in S. 
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Theorem 13.111 yields a powerful technique to analyze Tig. Using it, we prove the following two 
results. 

Theorem 14.61 Suppose p is an odd prime, g > ^, and < f < g. Suppose C is a generic hyperelliptic 
curve of genus g and p-rank f with Jacobian X = Pic'^(C). Then End(X) = Z and thus X is simple. 

The third theorem requires some notation. Let S be a connected stack over k, and let s be a 
geometric point of S. Let C — > S be a relative smooth proper curve of genus g over S. Then 
Pic'^(C)[£] is an etale cover of S with geometric fibers isomorphic to (Z/£)'^^. The fundamental 
group 7ti{S,s) acts linearly on the fiber Pic°(C)[£]s, and the monodromy group Me{C S,s) 
is the image of 7ti{S,s) in Aut(Pic'^(C)[£]s). For the third main result, we determine M^(S) := 
Mi{C S, s), where S is an irreducible component of Hg and C ^ S is the universal curve. 

Theorem I5.2ll5.7i Suppose p is an odd prime, g > 1, and < f < g- Let S be an irreducible component 

ofn{. 

(i) lfl< f < g and £ is an odd prime distinct from p, then Mf{S) = Sp2^{'^/i)- 

(ii) Iff = and if g > 4, then (S) = Sp2g(Z/£) /or all primes £ outside a finite set which depends 
only on p. 

We also prove that the £-adic monodromy group is Sp2g{'^i) in the situation of Theorem l5.2l/l5.7l 
(Note that the case of ordinary hyperelliptic curves, i.e., when f = g, follows directly from pre- 
vious work, see J.K.Yu [unpublished], IIAP071 Thm. 3.4], or llHal06l Thm. 5.1].) In addition, we 
determine the p-adic monodromy group of components of Ti.g when / > 1 (Proposition |5.4| ). 

This paper is a natural generalization of our paper IIAP08L which is about Mg, the p-rank strata 
of the moduli space of curves. The two papers share essential similarities, but there are several 
new phenomena for hyperelliptic p-rank strata which increase the difficulty of the proofs and 
influence the final results in this paper. First, the boundary component Aq is more complicated 
for Hg than for Aig- Second, for a singular hyperelliptic curve which is formed as a chain of 
two hyperelliptic curves of smaller genera, the set of possibilities for the location of the ordinary 
double point is discrete. These two facts play a key role in the degeneration arguments found in 
the proof of Theorem 13.111 The third issue, which arises in a base case when / = 0, is that the 
stratum Ti.'^ is not nearly as well understood as ^^3. 

The second and third main results of the paper rely on Theorem 13.111 because the proofs use 
degeneration in order to proceed by induction on the genus. For the inductive step. Theorem 

13.111 implies that the closure of every component S of Hg in Hg intersects the stratum Ai 1 of 
the boundary of Hg (Corollary I3.14|) . This is used in the proof of Theorem 14.61 to show that the 
endomorphism ring of Pic'^(C) acts diagonally on the Tate module. It is also used in the proof 
of Theorem l5.2l/l5.7l to show that the monodromy group of S contains two non-identical copies of 

Sp2g-2i^/i). 

There are two base cases needed in this paper. The first, when g = 2 and / > 1, uses facts 

about h{ from a special case of [ChaOSl Prop. 4.4]. The second, when g = 3 and / = 0, we found 
somewhat intractable. For this reason, we employ a novel analysis of endomorphism algebras of 
generic Jacobians of small genus with p-rank zero. Applying Theorem|3lI]and IIOor91l Thm. 1.12] 
lets us constrain the possibilities (Lemma 14. 1|) for the endomorphism algebra of the Jacobian of a 
generic hyperelliptic curve of genus 3 with p-rank 0. This, combined with an understanding of 
abelian varieties of Mumford type, yields sufficient leverage to understand the case when ^ = 4 
and / = 0. In particular, this allows us to determine the endomorphism ring of the Jacobian of 
a generic hyperelliptic curve of genus 4 and p-rank and to compute the monodromy group of 
components of Hi- 
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This paper also contains multiple applications about hyperelliptic curves of given genus and 
p-rank. For example, results on Newton polygons of (hyperelliptic) curves for arbitrary g and p 
are notoriously elusive. As a consequence of Theorem l3.11l we give an application about Newton 
polygons of hyperelliptic curves. The application relies on and generalizes [Oor911 Thm. 1.12], 
which is the case g = 5. 

Corollary |3.16[ Suppose p is an odd prime and g > 3. Let S he an irreducible component of Hg, the 
p-rank stratum in Tig. Then S contains the moduli point of a curve whose Jacobian is not supersingular. 

We also give applications about class groups and zeta functions of hyperelliptic curves of given 
genus and p-rank over finite fields. These build upon |KS99l Thm. 9.7.13] and |Kow06 l Thm. 6.1]. 
Applications: Let F be a finite field of characteristic p. Under the hypotheses of Theorem l5.2l/l5.7l 

(i) there is a hyperelliptic F-curve C of genus g and p-rank / whose Jacobian is absolutely 
simple (Application l5.9|) ; 

(ii) if |F| = 1 mod i, about — 1) of the hyperelliptic F-curves of genus g and p-rank / 
have a point of order £ on their Jacobian (Application 15.111) : 

(iii) for most hyperelliptic F-curves C of genus g and p-rank /, the splitting field of the numer- 
ator of the zeta function of C has degree 2Sg[ over Q (Application 15.13]) . 

Here is an outline of the paper. Notation and background are found in Section |2l Section |3] 

contains the results about the boundary of the p-rank / stratum Hg and the application to Newton 

polygons. This section ends with some open questions about the geometry of Hg. For example, 

the number of irreducible components of Tig is known only in special cases. The results about 
endomorphism rings are in Section H] while the results about monodromy and the applications to 
absolutely simple Jacobians, class groups and zeta functions are in Section |5l 

We thank the referee of [ AP08] for suggestions which we used in this paper and thank F. Oort 
for helpful comments. 

2. Background 

Let k be an algebraically closed field of characteristic p > 3. With the exception of Section l5!4l 
where we work over a finite field, all objects are defined on the category of fc-schemes. Let ^ be an 
odd prime distinct from p. We fix an isomorphism Hf = Z/£. 

2.1. Moduli spaces. For a natural number g consider the following well-known categories, each 
of which is fibered in groupoids over the category of A:-schemes in the etale topology: 

• Ag principally polarized abelian schemes of dimension g; 

• Aig smooth connected proper relative curves of genus g; 

• Hg smooth connected proper relative hyperelliptic curves of genus g; 

• M.g stable curves of genus g. 

Each of these is a smooth Deligne-Mumford stack, and Aig is proper |DM69l Thm. 5.2]. There 
is a natural inclusion Hg Aig) let Hg he the closure of Hg in A^^. Thus there are the following 
categories: 

• Hg stable hyperelliptic curves of genus g; 

• Hg stable hyperelliptic curves of genus g, along with a labeling of the smooth ramification 
locus (see llAFOTi Section 2.2]). 

Again, these are smooth proper Deligne-Mumford stacks IIEke95l Thm. 3.2], see e.g. |AF07l Lemma 
2.2] for Hg. The forgetful map CDg : Hg — > Hg is etale and Galois, with Galois group Sym(2^ + 2). 
If S C Hg, let S be the closure of S in Hg. Let Cg be the universal curve over Hg, i.e., the puUback 
to Hg of the universal curve over Aig (cf. [DM69 1 Thm. 5.2]). 
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For a natural number r, let Mg-r be the Deligne-Mumford stack of stable curves of genus g 
with r marked points. Let Hg-r = Hg x-^^ -Mg;,- and let Hg-j = Hg x-^^ -Mg-,,-- The forgetful map 
<pg;r '■ Hg-j Hg is proper, flat and surjective with connected fibers, and thus is a fibration. Let 

Hg;r = Hg-r x^^^ Hg. 

2.2. Stratifications. Let X be a principally polarized abelian variety of dimension g defined over 
k. There is an integer /(X) between and g, called the p-rank of X, such that X[p] {k) = {'L/p)f^^\ 
More generally, if X/fc is a semiabelian variety, then its p-rank is dimp^ Hom(^p,X). The p-rank 
of a curve is that of its Jacobian. If X — > S is a semiabelian scheme over a Deligne-Mumford 
stack, then there is a stratification S = US^ by locally closed substacks such that s € Sf(k) if and 
only if f{Xs) = f (this follows from IIKat79l Thm. 2.3.1], see, e.g., llAPOSl Lemma 2.1]). Thus, H^ 

is the locus in Hg parametrizing hyperelliptic curves of p-rank /. Every component of Hg has 
dimension g — '^+f IGPOSt Thm. 1]. 

Here is the definition of the Newton polygon of an abelian variety; see |Dem72l Chap. IV] 
for details. The isogeny class of a p-divisible group G/k is determined by 'v{G), a lower-convex 
polygon in connecting (0,0) to (height(G), dim(G)) with slopes /\ S Q fl [0, 1] and integral 
breakpoints. If X/Zc is an abelian variety, its Newton polygon is that of its p-divisible group X[p°°]. 
The Newton polygon is a finer invariant than the p-rank; indeed, the p-rank of X is exactly the 
length of the slope part of the Newton polygon. For example, X is ordinary exactly when its 
Newton polygon only has slopes and 1. By definition, X is supersingular if its Newton polygon 
only has slope 1/2. 

2.3. The boundary of the moduli space of hyperelliptic curves. The boundary of Hg is ^Hg = 
Hg - Hg. The description here of dHg follows [CHSS', Sec. 4(b)] closely In fact, while I CH88J is 
written for the base field C, the description of Hg and Hg is valid in any characteristic l|Yam04[ . 
Briefly, the irreducible components of (^Hg come from restriction of the components of the bound- 
ary of Mg, except that Aq breaks into several components. The informal discussion in this section 
is supplemented with precise definitions in Section l2!4l 

If ^ > 2, the boundary (^Hg is the union of components A, = A, \Hg] for \ < i < g — \ and 

= E:, [7?g] for < z < ^ - 2 by IIYam04l p.410]. Here A, and A^_, are the same substack of Hg 
and and are the same substack of Hg. Each A, and is an irreducible divisor in Hg. 

For 1 < z < g — 1, if n is the generic point of A,, then the curve Cg^r\ is a chain of two smooth ir- 
reducible hyperelliptic curves Yi and Y2, of genera i and g — i, intersecting in one ordinary double 
point P. The hyperelliptic involution t stabilizes each of Y\ and Y2. The point P is a ramification 
point for the restriction of t to each of Yi and Y2 but is not part of the smooth ramification locus. 

If r\ is the generic point of Eq, then the curve Cg^^i is an irreducible hyperelliptic curve self- 
intersecting in an ordinary double point P. The normalization of Cg^^j is a smooth hyperelliptic 
curve Yi of genus g — ^ and the inverse image of P in the normalization consists of an orbit under 
the hyperelliptic involution. 

For 1 < f < ^ — 2, if 77 is the generic point of E,, then the curve Cg^^j has two components Yi 
and Y2, which are smooth irreducible hyperelliptic curves, of genera i and g — 1 — i, intersecting 
in two ordinary double points P and Q. The hyperelliptic involution t stabilizes each of Yi and Y2. 
The points P and Q form an orbit of the restriction of t to each of Yi and Y2. 

One can associate to a stable curve C its dual graph, in which the vertices are in bijection with 
the irreducible components of C and in which there is an edge between two vertices exactly when 
the corresponding components intersect. A component of C is called terminal if the corresponding 
vertex is a leaf of the dual graph. A curve is called a tree if its dual graph is a tree. A curve is called 
a tree of elliptic curves if it is a tree and if each of its irreducible components is an elliptic curve. 
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A stable curve is a tree if and only if its Picard variety is represented by an abelian scheme; 
such a curve is also said to be of compact type. Let Aq = Ao['Hg] be the union of for < i < 
[(^ — l)/2j . The moduli points of stable hyperelliptic curves which are not of compact type are 
exactly the points of Aq [Hg] . 

2.4. Clutching maps. Recall from ||Knu83l that there are three types of clutching maps for positive 
integers gi and g2- 

'^gl.gl ■ '^gl ^ ''-g2 '^gl+gl '^gl+g2' 



'^gi '■ ^grA ^ ^gi- 



Each of these clutching maps is the restriction of a finite, unramified morphism between moduli 
spaces of labeled curves IIKnu83l Cor. 3.9]. One defines: 

^^i[^^i+g2] = Im(fCgj,gJ; ZoiUg.+i] =lm{Kg^)} and ^g^lUg.+g^+i] = lm{Ag^,g^). 

If S is a stack equipped with a map S Hg, let Aj [S] denote S x^^ Ai[T-Cg]. In particular, A, [7^g] = 

Hg Xjj^ Aj. Also define AilHgY := {Ai[Hg])f. Similar conventions are employed for E,-. 

These clutching maps can be described in terms of their images on T-points for an arbitrary 
fc-scheme T. 

2.4.1. Information about Kg^^g^. For i = 1,2, suppose S; G Hg.{T) is the moduli point of a hyper- 
elliptic curve y, with labeled smooth ramification locus. Then (^1/ S2) is the moduli point of 
the labeled T-curve Y where Y has components Yi and Y2, and where the last ramification point 
of Yi and the first ramification point of Y2 are identified in an ordinary double point. This nodal 
section is dropped from the labeling of the ramification points; the remaining smooth ramification 
sections of Yi maintain labels {1, . . . , 2gi + 1} and the remaining ramification sections of Y2 are 
relabeled {2gi + 2, . . . , 2{gi + gi) +2}. There is a unique hyperelliptic involution on Y which re- 
stricts to the hyperelliptic involution on Yi and on Y2. Moreover, Kg^ g^ (^1/ Si) is the moduli point 
of the (unlabeled) hyperelliptic curve Y. 
By IIBLR901 Ex. 9.2.8], 

(2.4.1) Pic°(Y) ^ Pic°(Yi) x Pic°(Y2). 
Then the p-rank of Y is 

(2.4.2) f{Y)=f{Y,)+f{Y^). 

2A.2. Information about Kg^. Suppose si G Hgy^i{T) is the moduli point of (Yi;P), a hyperelliptic 
curve with a marked section. Then Kgj (si) is the moduli point of the T-curve Y, where Y is the 
stable model of the curve obtained by identifying the sections P and t(P) in an ordinary double 
point P'. The hyperelliptic involution on Yi descends to a unique hyperelliptic involution on Y. 
By IIBLR901 Ex. 9.2.8], Pic°(Y) is an extension 

(2.4.3) Z Pic° ( Y) Pic° ( Yi ) , 
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where Z is a one-dimensional torus. In particular, the toric rank of Pic (Y) is one greater than that 
of Pic°(yi), and their maximal projective quotients are isomorphic, so that 

(2.4.4) /(Y)=/(y,) + l. 

2.4.3. Information about ^gi,g2- For / = 1,2, suppose s, G T-Lg.-i{T) is the moduli point of (Y,;P,), 
a hyperelliptic curve with a marked section. Then -^^1,^2(^1' ^2) is the moduli point of the stable 
model y of the T-curve obtained by identifying Pi and P2 in an ordinary double point P and 
by identifying t(Pi) and t(P2) in an ordinary double point Q. There is a unique hyperelliptic 
involution on Y that restricts to the hyperelliptic involution on Yi and on Y2. 

By IIBLR90 i Ex. 9.2.8], Pic°(Y) is an extension 

(2.4.5) Z Pic°(Y) Pic°(Yi) x Pic°(Y2) , 

where Z is a one-dimensional torus. In particular, 

(2.4.6) /(D)=/(Yi)+/(Y2) + l. 

For later use, here is a description of the stable model Y when Pi is a ramification point of Yi, 
but P2 is not a ramification point of Y2. Then Y consists of three components, namely the strict 
transforms of Yi and Y2 and an exceptional component W which is a projective line. Also Yi 
intersects W in an ordinary double point and Y2 intersects W in two other points, which are also 
ordinary double points. 

2.4.4. Clutching along trees. The definition of Kg^^g^ above relies on an arbitrary, although conve- 
nient, choice of sections along which to glue, and a labeling of the smooth ramification locus of 
the resulting curve. By considering morphisms of the form 7gi+g2 ° '^g\,g2 ° {^gi ^ 'Ygi)' where 
ygi+gi ^ Sym(2(gi + ^2) + 2) and y^j G Sym(2gi + 2) and jg^ G Sym(2^2 + 2), it is possible to 
clutch along arbitrary sections, with complete control over the subsequent labeling. 

This can be used to describe configurations of curves of compact type, as follows. A clutching 
tree is a finite tree A along with a choice of natural number gi, for each vertex y G A such that 
deg(w) < Igz, + 2. Such a tree is called a clutching tree of elliptic curves if gj, = 1 for all u G A. Let 
I A| be the number of vertices in A, and let g(A) = Y^vga gv- 

Using a product of the clutching maps defined above, one can define a morphism 

Let A/v be the image of ka- If rj is the generic point of Aa, then Cg^r, is a hyperelliptic curve 
of compact type with dual graph isomorphic to A, such that the irreducible component of Cg^ri 
corresponding to the vertex v has genus gv 

Suppose Vi and V2 are adjacent vertices in a clutching tree A. Consider the tree A' obtained by 
identifying Vi and V2 in a new vertex, v, which is adjacent to all neighbors of Vi or V2 in A, with 
label g-o = g-o^ + gv2 ■ Then ka factors through ka' and Aa C Aa' . A tree A refines a tree A' if A' 
can be obtained from A through iterations of this construction. 

2.4.5. One more clutching map. In the special case where A is a tree on three vertices, where the 
leaves Vi and 1^3 have gv^ = gv^, = 1/ arid where gv2 = S~ 2, one obtains the following diagram: 



(2.4.7) 



Til X ^ ^1 



'Ha-i X Til 



Til X Tig-i 
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Let Ai 1 \Hg] be the image of ki g_2,i = cOgO ki g_2,i; it is an irreducible component of the self- 
intersection locus of Ai \Hg]. If r\ is the generic point of A14 \Hg\, then the curve Cg^ri is a chain of 
three smooth irreducible hyperelliptic curves Yi, Y2, Ys with gY^ = gy^ = 1 and gy^ = g — 2. For 
z G {1, 3}, the curves Y, and Y2 intersect in a point P, which is an ordinary double point. 

3. Boundary of the hyperelliptic p-rank strata 

3.1. Preliminary intersection results. Suppose p > 3, and g > ^ and < f < g. The p-rank 
strata of the boundary of Hg are easy to describe using the clutching maps. First, H I < i < g — 1, 
then (I2.4.2I 1 implies that AjlHgY is the union of the images of x 'H^l, under K;,^-! as {firfi) 
ranges over all pairs such that 

(3.1.1) 0<f^<i,0<f2<g-i and /i + /2 = /. 

Second, if / > 1, then ^QlHgY is the image of Hg_i-i under Kg_i by (|2.4.4|) . Third, if / > 1 and 

1 < i < g — 2, then (I2.4.6I I implies that EljlHgY is the image of h{.i x Ti-^g^i^i-i under A,-g_i_, as 
(/1//2) ranges over all pairs such that 

(3.1.2) < /i < f, < /2 < g - 1 - z and /i + /2 = / - 1. 
Lemma 3.1. Suppose g > 2 and < f < g. 

(a) Ifl<i<g — 1, then every component of /S.i\HgY and of /S.\\HgY has dimension g — 2 + f. 

(b) If f > 1 and < i < g — 2, then every component ofEi^J-LgY and ofZi^hCgY has dimension 

Proof. For parts (a) and (b), the claims for Tig and for Tig are equivalent, since cOg is a finite map 

which preserves the p-rank stratification. Recall that and Tig are pure of dimension g — 1 + / 
iGPOSl Thm. 1]. 

For part (a), suppose < f < g, and 1 < i < g — 1, and that {fi, fi) is a pair which satisfies 
(ISTTl) . Then Tif x Ti^^l^ is pure of dimension dim(7if ) + dim(7^(i,.) = g-2 + f. Since k/,^_, is 
finite, Ai^gY is pure of dimension g — 2 + / as well. 

For part (b), first suppose i = 0. Then HgYl-i is pure of dimension dim{Hg_\) + 1 = g — 2-\- f. 
Since Kg_i is finite, [HgY is pure of dimension ^ — 2 + / as well. 

To finish part (b), suppose 1 < i < g — 2, and that (/i,/2) is a pair which satisfies (|3.1.2|l . Then 
^{1 ^ is Pui's of dimension dim(7^f ) + dim(7i^'_._-^) + 2 = ^- 2 + /. Since Ai,g-i-i is 

finite, ^iiUgY 

is pure of dimension g — 2 + / as well. □ 

The next lemma shows that if t] is a generic point of Hg, then the curve Cg^r] is smooth. Thus no 
component of Hg is contained in the boundary dHg. 
Lemma 3.2. Suppose g > 1 and < f < g. 

(a) Then Hg is open and dense in H^^ and Hg X:^ Hg is open and dense in Hg. 

f — f ~ f ~ f 

(b) Ifr> 1, then Hg-j is open and dense in Hg.,. and Hg x^ Hg is open and dense in Hg. 

Proof. Part (a) is well-known for ^ = 1. For g > 2, part (a) follows from Lemma |3H since H^^ and 
Hg are pure of dimension ^ — 1 + / IIGP051 Thm. 1]. Part (b) follows from part (a) since the p-rank 
of a labeled curve depends only on the underlying curve, so that Hg.r = ^g;r '^i- ^ 
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Suppose S is an irreducible component of Hg and let S be its closure in Hg. Note that S can 
contain the moduli points of curves with lower p-rank. (In fact, this always happens when / > 1, 
see Corollary 13.151 ) 

Lemma 3.3. (a) Let S be an irreducible component ofHg. If S intersects a component T of Ai[Hg]f 
then^S contains P. _ _ _ 

(b) Let S be an irreducible component ofHg, with closure S*.IfS* intersects a component V o/A, [HgY, 
then S* contains V. 

Proof. Part (a) is proved here; part (b) is proved in an entirely analogous fashion. A smooth proper 
stack has the same intersection-theoretic properties as a smooth proper scheme |Vis89i p. 614]. 
In particular, if two closed substacks of Hg intersect then the codimension of their intersection 
is at most the sum of their codimensions. Now S and Ai[Hg] are closed substacks of Hg. By 
Lemma l3!2l a), S ^ Ai[Hg]. Thus the intersection of S with the divisor A, [7^g] has pure dimension 
dim S — 1, which equals dim(A,[7^g]-^) by Lemma |3.1[ Thus if S intersects a component F of 
A, [HgY then it must contain the full component F. □ 

Lemma 3.4. Suppose g > 2 and < f < g. Let S be an irreducible component ofHg. 
(a) Then S intersects Ao[Hg] if and only iff > 1. 

(h) If f > 1, then each irreducible component of Aq[S] contains either (i) the image of a component 
ofH^_i.i under Kg_i or (ii) the image of a component ofHi^i x Hg_i_i.i under Aj^g^i^jfor some 
1 < i < g — 2 and some pair (firfi) which satisfies (|3.1.2|) . 

(c) If f = 0, then S contains the image of a component ofH'- x under Kj^g^ifor some 1 < i < 

Proof If / = 0, then (|2A4l) and ( |2A5l) imply that S doe s not in tersect Aq [Hg] . If / > 1, then S is a 
complete substack of dimension greater than g — l.By ]FvdG04. Lemma 2.6], a complete substack 
of Hg — Ao[Hg] has dimension at most g — l- Therefore, S intersects Ao['Hg]. This completes part 
(a). _ __i _ 

For part (b), each irreducible component of Ao[S] intersects either '<g-i(Wg-i;i) C H;o[7^g] or 

Ai,g-i-i{H{}i X H^g^i_i.i) C H;;['Hg] for some 1 < / < ^ — 2 and some pair (/i,/2) which satisfies 
(|3.1.2|) . The result then follows from Lemma l33l a). 

For part (c), recall that Hg contains no complete substacks of positive dimension (e.g., IIYam04l 
Cor. 1.9]). Thus S intersects Aj for some < z < g — 1. By part (a), i ^ 0. The result follows from 
Lemma IS^t a). □ 

3.2. Complements on trees. Many of the results for A, for positive / have analogues for A^. For 
a clutching tree A and a nonnegative integer /, define an index set by 

(3.2.1) J^{A,f) = |{/, : t; e A} : < /. < = /| . 

Lemma 3.5. Let Abe a clutching tree with g{A) = g. 

(a) The p-rank strata of A/^\Hg\ are given by 

(3.2.2) AA[7?g]/= U KA(x,eA7i{:). 

{/„}e.^(A,/) 

(b) Every component of Aj^\Hg\f has dimension g + f — \ A\. 
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Proof. Part (a) follows from (|3.1.1|) and induction on |A|. Part (b) follows from this and the calcu- 
lation that, for {/i,} e J^(A,/), 

dim{x,^Ant) =Z{gv+fv-l)=g + f- \A\. 

veA 

□ 

Lemma 3.6. Let S be an irreducible component ofHg. Let Abe a clutching tree with g{A) = g. If S 
intersects a component V of A/\^[Hgy , then S contains V. 

Proof. The proof is similar to that of Lemma l33l a). Note that dim P > dim S + dim A a — dim Tig. 
By Lemma l33l:b). this equals g + f - \ A\ = dim ( A a [Hg] ^ ) . □ 

3.3. Adjusting marked points and trees. The next lemma shows that one can adjust the marked 
points of an r-marked hyperelliptic curve of genus g and p-rank / without leaving the irreducible 

component of Hg.,. to which its moduli point belongs. 

Lemma 3.7. Let S be an irreducible component of 'Hg-r, ^md let S be the closure of S in Hg.,-. Then 
S = (pg.j:{(pg;r{S)). Equivalently,ifT is ak-scheme,if (C;Pi, . . . ,Pr) G S{T), andif {Qi, . . . ,Qr) is any 
other marking ofC, then (C; Qi, . . . , Qr) G S(T). 

Proof. It suffices to show that (pg^.{(pg-r{S)) C S. Note that S is the largest irreducible substack of 
H^g-r which contains S. The fibers of 4>g;r\s are irreducible, so (l)g.j{(pg-r{S)) is also an irreducible 
substack of Ti-g-f which contains S. Thus cpg.J.{(pg-r{S)) C S. This shows that (pg.J:{(j)g-r{S)) = S. 

To finish the proof, it suffices to show that the T-points of S and 0~.J(0^;^(S)) coincide for an 
arbitrary fc-scheme T. To this end, let a = (C; Pi, . . . , Pr) E S(T), and let /3 = (C; Qi, . . . , Q,-) G 
Hg.^{T). Note that (pg-r{l^) = 4>g;r{oc), and (pg-ri^x) is supported in the closure of (/)g;r(S) in Hg. By 
Lemma l3^ b), Hg;r is dense in Hg;,-. It follows that (3 is supported in the closure of 4)^.,^(0g;r(S)) 
in Hg.j, which is S. □ 

It is not clear whether one can change the labeling of the smooth ramification locus of a hyperel- 
liptic curve without changing the irreducible component of Hg to which its moduli point belongs. 
To circumvent this issue, the following lemma about hyperelliptic curves of genus 2 and p-rank 1 
will be useful. 

Lemma 3.8. (a) First, h\ is irreducible and intersects ki^i{H\ x H'(). 

(b) Second, let S be an irreducible component ofH\. If S intersects Hi^ilHl x H^), then S also mfer- 
sectsKi,i{H^ xHl). 

Proof. For part (a), recall that the Torelli morphism H2 A2 is an inclusion IIOS801 Lemma 1.11]. 
Now dim(7^i) = dim(^^) and Al is irreducible (e.g., EvdG09l Ex. 11.6]). It follows that Hi 

is irreducible and thus h\ is irreducible by Lemma I3.2r a). Consider a chain Y of two elliptic 
curves, one ordinary and one supersingular, intersecting in an ordinary double point, which is a 
fixed point of the hyperelliptic involution on each elliptic curve. The moduli point of Y is in the 

intersection of K14 {H\ x Hi) and 7^2- 

For part (b), let S* be the closure of S in 7^2- By hypothesis and Lemma l33l b), S contains a 
component ofKi^i{H\ x H^). Every component of H\ contains a component of H^ in its closure 
since any nonisotrivial proper family of curves of genus one has supersingular fibers. It follows 
that S* contains a component of ki4(7^5 ^ ^1)' thus intersects the closure of a component of 
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X Hi). By Lemma l33l b), S* contains a component of ki 1(7^5 ^ ^1)' which then implies 
the same for S. □ 

Remark 3.9. Note that a genus two curve has six ramification points and thus there are potentially 
up to 6! = |Sym(6)| irreducible components of H\. In particular, the fact from Lemma l3^ a) that 
Ti.\ intersects ki x H^) does not imply the hypothesis in part (b) that S intersects ki i('H| x 

Lemma 3.10. Let S be an irreducible component ofHg- Sup-pose A is a clutching tree of elliptic curves 
with g{A) = g. If S intersects A/<^[Hg], then for any choice of {fz,} G J^{A,f), S contains an irreducible 
component of k/\{x vGA^gl ) . 

Proof. By Lemma l33l a), for some choice of data {/*} G T{A,f), the intersection of S and Aj\\Hg\ 

contains a point of K/\{Xy^/(Hg[,). By Lemma l3!6l there are components Ty C 1-Lg[, such that S 

contains k\{x-o^/\T-o). One immediately reduces to the case in which vi and V2 are adjacent vertices 
in A with f*^ = 1 and f*^ = 0, and {fv} G J^{A,f) is given by 

fo V^{V^,V2} 
^1-/* ve{Vi,V2}. 

Let A' be the tree obtained by identifying vi and f 2 in a new vertex V12 with gi,^^ = 2. By Lemma 
13.61 there is a component T^j^ C H\ such that S contains 

Now, Ti,j2 contains a component of ki i (7^1 x Hi). By Lemma l3.8f b), Tjj^^ contains a component of 
Ki 1 (7^° X 7^ J ) as well. Then S contains a component of ka ( x -o^AHgl ) . □ 

3.4. Main intersection result. In this section, we prove that the closure of each irreducible com- 
ponent S of Tig contains the moduli point of a singular curve which is a tree of elliptic curves and 
has p-rank /. 

Theorem 3.11. Suppose g >2 and < f < g. Let S be an irreducible component of Tig. 

(a) There exists a clutching tree of elliptic curves A with g{A) = g such that S contains an irreducible 
component of A/\\J-Lg\f . 

(b) For any choice of {fv} G !F{A, f), S contains an irreducible component of K/\^{Xy^/(Hg'^). 

(c) In particular, S contains the moduli point of some tree of elliptic curves, of which f are ordinary 
and g — f are supersingular. 

Proof. It suffices to prove part (c) since parts (a) and (c) are equivalent by Lemma 13.61 and since 
parts (a) and (b) are equivalent by Lemma 13.101 

First suppose g = 2. If / = 2, then H2 is irreducible and affine and H2 contains the moduli 
point of a tree of 2 ordinary elliptic curves. If / = 1 (resp. / = 0), the result is true by Lemma 
21Ia) (resp. Lemma I3.4f c)). Now suppose g > 3 and < f < g and suppose as an inductive 



hypothesis that the result is true when 2 < g' < g. Let S be an irreducible component of Hg. 

Claim 3.12. To complete the proof, it suffices to show that S intersects A,[7^g]-^/or some 1 < i < g — 1. 

Proof of claim. Suppose S intersects A, [7^g]^ for some 1 < i < g — 1. Let gi = i and gi = g — i- 
By Lemma l33l a). S contains a component of Ag^ \HgY . In other words, there exist a pair (/i,/2) 
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satisfying (I3.1.1D and, for j = 1,2, components Vj of HJ^ such that S contains Kg^^g^ (^i ^ ^2)- Then 
Vj = CDg.{Vj) is a component of Hg.. 

By the inductive hypothesis, Vj contains the moduli point Sj of a tree Yj of gj elliptic curves, of 
which fj are ordinary and gj — fj are supersingular. Let Ay be the dual graph of Yj. Let 'sj € Vj be 
such that £Pg. (sy) = Sy. In other words, Sy is the moduli point of Yy along with the data of a choice 
of labeling of the smooth ramification locus. Then Kgi,g2(si7^2) is the moduli point of a curve C 
whose dual graph is obtained by connecting a vertex of Ai with a vertex of A2. Since C is a tree, 
( Si / S2 ) is the moduli point of a tree of g elliptic curves, of which f = fi+ fi are ordinary and 
g — f are supersingular. This completes the proof of the claim since Kgi,g2 (si, S2) is in S. □ 

Continuing the proof of Theorem 13. Ill first suppose / = 0. By Lemma I3.4r c), S intersects 

some 1 < / < ^ — 1. By Claim l3A2l this completes the proof when / = 0. 
Now suppose / > 0. By Lemma l3!4l a), S intersects Ao[T-CgY. 

Case (i): S intersects Eq. 

By Lemma l3!4l b), S contains the image of a component v' of Hg_\.i under Kg-i. Consider 

V = cj)g^i-i{v') which is a component of 'Hg_\. By the inductive hypothesis, V contains the 
moduli point of a curve Yi which is a tree of g — 1 elliptic curves, of which / — 1 are ordinary and 
g — f are supersingular. Let E be a terminal component of Y-[ and let Y[ be the closure of Yj — E 
in Yi . Let R be the point of intersection of E and Y[ . Since the quotient of Yi by the hyperelliptic 
involution t has genus 0, the elliptic curve E is stabilized by t. Let P 7^ R be a point of E which is 
not a ramification point of t. By Lemma l3!7l the moduli point i of (Yi; P) is in y'. 

Let Z be the singular irreducible hyperelliptic curve of genus two with exactly one ordinary 
double point P' such that the normalization of Z is the elliptic curve E and the pre-image of P' 
consists of the points P and t(P). In other words, the moduli point of Z is the image of the moduli 
point of (E; P) under ki. 

Consider the point s = Kg_i(f) of S. The curve Cg^s has components Z and Y[ which intersect 
in exactly one ordinary double point, R. The p-rank of Cg^s is /(E) + f{Y[) + 1 = /. Since > 3, 
there is a terminal component of Yj not containing R which is an elliptic curve. Thuss G AilUgY- 
(In fact, s is also in /S.2\Hg]f because of the component Z.) By Claim 13^121 this completes Case (i). 

Case (ii): S intersects for some \ <i < g — 1. 

Let ^1 = / and g2 = g — f — i- Lemma l3!4l b), S contains a component of 'Ei[HgY . In other 
words, there exists a pair [fi, fi) satisfying (|3.1.2|) and, for ; = 1,2, there exist components v'j of 
'H^g.-i such that S contains Ag^^g^{v'i x v'2). Then V j = 4)g..i{v'j) is a component of Ti!^.. 

By the inductive hypothesis, V j contains the moduli point Sy of a tree Yy of gj elliptic curves, 
of which fj are ordinary and gj — fj are supersingular. Let Ey be a terminal component of Yy and 
let Y'j be the closure of Yy — Ey in Yy. Let Ry be the point of intersection of Ey and Y'-. Since the 
quotient of Yy by the hyperelliptic involution ty has genus 0, the elliptic curve Ey is stabilized by ty. 
Let Pi 7^ Ri be a point of Ei which is a ramification point of ti . Let P2 7^ R2 be a point of E2 which 
is not a ramification point of 12- 

By Lemma [371 the moduli point s'- of {Yj;Pj) is in v'j. Consider s = ^gi,g2(^'i'^2) which is a 
point of S. By Section l2.4.3[ the components of the stable model Cg^s are the strict transforms of Yi 
and Y2 and an exceptional component W which is a projective line. Moreover, Yi intersects W in 
an ordinary double point and Y2 intersects W in two other points, which are also ordinary double 
points. The p-rank of Cg^s is /(Yi) + /(Y2) + 1 = /■ 
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The curve Cg^s has a terminal component E'^ of genus 1. To see this, when i = 1 let E[ = E-[, and 
when i > 1 let E'^ ^ Ex be another terminal component of Y\. It follows that s is in Ai \H^^ ■ (Also 
s is in A, ['Hg]^ because of the component Yi.) By Claim 13^21 this completes Case (ii). □ 

3.5. Three corollaries. Here are several consequences of Theorem |3. 1 1 1 which will be used later in 
the paper. 

In the setting of Theorem l3.11[ one can deduce that S intersects A, nontrivially only when A has 
an edge whose removal yields two trees of size i and g — i- This is only guaranteed when i = 1. 
Luckily, the following information on degeneration to Ai is sufficient for the later applications in 
the paper. 

Corollary 3.13. Suppose g > 2 and < f < g. Let S be an irreducible component of Hg. Then S 
intersects Ai [HgY- Furthermore: 

(a) /// <§■ — !, then S contains an irreducible component o/Ki^g_i(7-^5 x Hg_^); and 

(b) iff > 1, then S contains an irreducible component of Ki^g_i{H\ x Hg_l). 

Proof. By Theorem 13 . 11 f c) , S contains the moduli point of a tree of elliptic curves, of which / are 
ordinary and g — f are supersingular. Every tree has a leaf; by Theorem 13. 1 1 f b) , that leaf can be 
chosen to be ordinary or supersingular if the obvious necessary constraint is satisfied. The result 
follows by Lemma |3^ □ 

The ^-adic and p-adic monodromy proofs in Section |5] rely on degeneration to A14. One can 
label the four possibilities for (/i,/2,/3) such that /i + /2 + /a = / and < /i,/3 < 1 as follows: 
(A) (1,/ - 2, 1); (B) (0,/ - 1, 1); (B') (1,/ - 1, 0); and (C) (0,/, 0). 

Corollary 3.14. Suppose g >3 and < f < g. Let S be an irreducible component ofHg. 

(a) Then S intersects Ai^ilHgY- 

(b) There is an irreducible component S of S x Hg, and a choice of [fi, fi, f 2,) from cases (A)-(C); and 
there are irreducible components Si ofHi and S2 ofHg^_2 '^^^ ^3 ofH{^; and there are irreducible 
components Sr of Ti^g^^^ and Sl of T-L^g^-l^; such that the restriction of the clutching maps of 
{1A.7) yields a commutative diagram 

(3.5.1) Sl X S2 X S3 ^ Sl X Sr 



Sl X S3 ^SnAi,i[ng\. 

(c) Furthermore, case (A) occurs as long as f > 2, cases (B) and (B') occur as long as 1 < f < g — 
and case (C) occurs as long as f < g — 2. 

Proof. By Theorem 13. Uf a), there is a clutching tree of elliptic curves A such that S contains a 
component of AAlHgY. Let Vi and W3 be two leaves of A; using Theorem 13. llT b). one can assume 
that fi,. = fi where \fi, fi, f?,) is chosen as in part (c). Let A' be the tree obtained by coalescing 
all vertices of A except for ^i and W3. Let V2 denote this new vertex and let f-,,^ = /2. Since A 
refines A', then S intersects A/\\HgY which completes part (a). Moreover, there is an irreducible 
component S of S X:^ Tig such that S intersects A.i^i\HgY . Part (b) follows from the definition of 
A14 [7^,,]/ and LemmaESl □ 

Corollary 3.15. Let S be an irreducible component ofHg. For each < f < f, there exists an irreducible 

f — 
component T ofHg such that S contains T. 
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Proof. It suffices to prove the result for /' = / — 1. Let S* be the closure of S in Hg — Ao[Hg]. 
A purity result IOor74l Lemma 1.6] shows that S* — (S*)^, if nonempty, is pure of dimension 
dim S* — 1. In particular, let Z = (S*)^^^; then Z, if nonempty, is pure of dimension g — 2-\- f. 

By Corollary |3.13[ b), S* contains an irreducible component of kj x H^^_l). Since H\ is 
dense in Hi, its closure contains the moduli points of supersingular elliptic curves (with labeled 
smooth ramification locus). Therefore, S* contains an irreducible component of ki^i{H^ x Hg_\), 

and Z is nonempty. Then dim Z = g — 2 + f = dim 7^^ ^, and so Z contains a component T of 
Hg ^. By Lemma l3^ a) S contains a component T of Hg ^. □ 

3.6. Application to Newton polygons. Recall that a stable curve C of compact type is supersin- 
gular if all the slopes of the Newton polygon of its Jacobian equal 1/2. This is equivalent to the 
condition that the Jacobian of C is isogenous to a product of supersingular elliptic curves. Note 
that a supersingular curve necessarily has p-rank zero. An abelian variety of p-rank zero is neces- 
sarily supersingular only when the dimension satisfies g < 2. 

In this section, we prove that the Newton polygon of a generic hyperelliptic curve of p-rank is 
not supersingular when g > 3. The result generalizes IIOor91 , Thm. 1.12] which is the case ^ = 3. 

Newton polygons have the following semicontinuity property: let S = Spec(K) be the spectrum 
of a local ring, with generic point rj and geometric closed point s; if G is a p-divisible group over 
S, then 'v{Gn) either equals or lies below t/(Gs). (The latter condition means that viGn) and v(Gs) 
have the same endpoints and all points of t^(G,j) lie below ■v{Gs).) 

Corollary 3.16. Suppose p is an odd prime and g > 3. Let rj be a generic point ofHg. Then Cg^j^ is not 
supersingular In particular, there exists a smooth hyperelliptic curve of genus g and p-rank which is not 
supersingular. 

Proof. When g = 3, this follows from IIOor91l Thm. 1.12]. For g > 4, the proof proceeds by 
induction. Let S be the closure of r\ in H^g. By Corollary 13. 131 S contains a component of Ai ['Hg]°. 

Thus there are components Vi of H\ and V2 of such that S contains Ki^g^iiVi 'xV2). By 

the inductive hypothesis, the Newton polygon of the generic point of V2 is not supersingular; in 
particular, it has a slope A such that < A < 1/2. The same is then true of the generic point of 
Ki^g-i{Vi X V2). By semicontinuity llKat79l Thm. 2.3.1], the generic Newton polygon of S (and 
thus of S) either equals or lies below that of {Vi x V2). In particular, it has a slope A' < 1/2. 
Thus Cg^r] is not supersingular. □ 

Remark 3.17. When p = 2 (a case not considered in this paper), there are some results about the 
slopes of Newton polygons of hyperelliptic curves of p-rank 0, see e.g. I|SZ02| . 

3.7. Open questions about the geometry of the hyperelliptic p-rank strata. 

Question 3.18. Does the closure of each component of Hg contain the moduli point of a chain of 
elliptic curves with p-rank /? 

If the answer to Question 13.181 is affirmative then Lemma 13.101 implies that every ordering of 
/ ordinary and g — f supersingular elliptic curves occurs for such a chain. In IIAF081 Cor. 3.6], 

the authors show the analogous question has a positive answer for every component of Mg. The 

difference for Mg is that the clutching morphism identifies two curves at an arbitrary point of 
each, rather than a ramification point of each. The location of these points can then be changed 
using an analogue of Lemma [321 

Question 3.19. For 1 < i < g — 1, does the closure of each component of Hg intersect Ai\Hg\fl 
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In |AF08l Prop. 3.4], the authors show that the analogous question has a positive answer for ev- 
ery component of Mg, also with control over the arrangement of p-ranks. An affirmative answer 
to Question 13.181 would imply an affirmative answer to Ouestion l3.19l 

Question 3.20. How many irreducible components does Hg have? 

One knows that Tig is irreducible for all p when f = g or when g = 2 and / = 1. If ^ > 3, then 

Ag is irreducible by IChaOSl Remark 4.7]. If Hg is irreducible, then there is a very short proof that 
Questions 13. 18l and l3.19l have affirmative answers. 

4. Endomorphism rings 

In this section, we use degeneration results from Section |3] to constrain the endomorphism ring 
of a generic curve of given genus and p-rank. 

Let Xg = Pic° be the neutral component of the relative Picard functor of Cg over Hg; then 

A!g — > Hg is a semiabelian scheme. To ease notation, if X is an abelian variety, let E(X) = 
End(X) d Q, and let Ef(X) = E(X) (g)Q Q^; then E^(X) acts the rational Tate module V({X) := 
T£{X) Qe- If X is simple, then the center of E(X) is either a totally real or totally imaginary 
number field. 

Lemma 4.1. Let £, be a geometric generic point ofH^. Then X^^^ is simple and either E^X^^^^) = Q or 
E(<^3,£) is isomorphic to a totally real cubic field L such that L^Qp is afield. 

Proof. Suppose there is an isogeny A's £ ~ Ai © A2 for abelian varieties Ai and A2 of dimensions 1 
and 2. Then Ai and A2 each have p-rank and are thus supersingular. Then X^^i^ is supersingular, 
which contradicts the fact that the Newton polygon of A's £ has slopes 1/3 and 2/3 10or91l Thm. 
1.12]. Thus ^"3 £ is simple. 

By the classification of endomorphism algebras of simple abelian varieties of prime dimension 
(e.g., |Oor88i 7.2]), to complete the proof it suffices to show that neither a complex multiplication 
field of degree six nor a quadratic imaginary field acts on A^a^^. Let S be the closure of ^ in Hy 
Since dim S = 2 > but abelian varieties with complex multiplication are rigid, E(A'3 £) is not a 
complex multiplication field of degree 6. 

To address the possibility of an action by a quadratic imaginary field K, suppose to the contrary 
that there is a subring of End(A'3 £ ) isomorphic to an order Ok in K. Then Ok ^ acts on the 
p-divisible group A'3,£[p°°]. There is an inclusion K(g)Qp'—> End(A'3,£[p~]) ^Qp = D1/3 © D2/3. 
(Here, D;^ denotes the central simple Qp-algebra with Brauer invariant A.) Every maximal subfield 
of or D2/3 is a cubic extension of Qp, but K^Qp is a Qp-algebra of degree two, so K^Qp 
cannot be a field. In particular, ^"3 £ does not admit an action by a quadratic imaginary field inert 
or ramified at p. 

Finally, suppose X^,^^^ admits an action by a quadratic imaginary field K which splits at p. Let 
(r, s) be the signature of the action of Ok on Lie(A:3 £); the dimensions r and s are nonnegative 
and r + s = 3. Consider the moduli space ShQ^.j^^ ,,-^ of abelian threefolds with an action by Ok of 
signature (f, s). The Torelli morphism t restricts to a finite morphism from S to a component of 
^^OK;(r,s)- Since dim S = 2 and dimShQ^.j^yg^ = r ■ s, then (r,s) is either (1,2) or (2, 1). Thus t(S) 
is dense in ShQ^.^^^^^y This gives a contradiction since Xo,^^^ has p-rank zero but the generic member 
of Sho^-{r,s) is ordinary IIWed99l Thm. 1.6.2]. 

Therefore, E(A'3^£) is either Q or a totally real cubic field, L. In the latter case, the p-rank zero 
locus in a Hilbert modular threefold attached to L has dimension (at least, and thus equal to) two, 
which forces L ® Qp to be a field. □ 

Remark 4.2. In the situation of Lemma [4^11 it is not known which outcomes occur. 
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Lemma 4.3. Let Y be a simple abelian variety whose dimension g is relatively prime to 3. If there exists a 
geometric generic point ^3 of for which there is a nontrivial homomorphism End(Y) ^{JY^^^^), then 
End(y) ^ Z. 

Proof. If Z is a simple abelian variety, let Eo(Z) be the subfield of E(Z) fixed by the Rosati involu- 
tion, and let eo(Z) = [Eo(Z) : Q]. Theneo(Z)| dim(Z). 

By Lemma [4.1[ E{A!3^£^^) is a totally real field of dimension 1 or 3 over Q. On one hand, the 
existence of a nontrivial homomorphism End(Y) E(A'3 .fg) forces eo{y) to divide eQ{X^^^^), and 
thus eo(y)|3. On the other hand, eo{Y)\g. Therefore, eo(^) = 1 and Eo(y) = Q. Neither a non- 
commutative algebra nor a totally imaginary field admits a nontrivial homomorphism to E(A'3 .f^), 
and thus E(y) - Eo(y) and End(y) ^ Z. ' □ 

Lemma 4.4. Let X ^ S be a polarized abelian scheme over a reduced irreducible Noetherian stack. Let r\ 
be the generic point of S, and let s ^ S be any point. Then there exists an inclusion End(X7j) ^ End(Xs). 

Proof. By introducing a rigidifying structure on X — > S, such as coordinates on the space of sec- 
tions of the third power of the ample line bundle given by the polarization, one can assume S is 
a reduced irreducible Noetherian scheme. Since the absolute endomorphism ring of an abelian 
variety is defined over a finite extension of the base field, it suffices to show the existence of an in- 
clusion End(X^) ^ End(Xs). If S is normal, then End(X^) extends uniquely to End(Xs), and in 
particular to End(Xs) l|FC90l 1.2.7]. In general, let "v : S' — > S be the normalization map; let rj' be the 
generic point of S', and let s' be a point of S' over s'. The desired result follows from the canonical 
map End(('K*X)^') ^ End((-v*X)s/) and the isomorphisms of abelian varieties ('v*X)^/ = X^ x 77' 
and {'v*X)s' = X^ x s'. □ 

Proposition 4.5. If £,4 is a geometric generic point of Til, then P^4,^ is simple and End(A:'4,^^) = Z. 

Proof. Let S4 be the closure of ^4 in Ti^. Suppose there is an isogeny X^^^^^ ~ Ai © A2 for two 
abelian varieties Ai and A2. If Ai and A2 each have dimension 2, then they are supersingular 
since they have p-rank 0. Then .^4 is supersingular, which contradicts Corollary 13.161 If Ai has 
dimension 1 and A2 has dimension 3, then there is a curve W of genus 3 such that Jac(W) = A2. 
The inclusion of A2 into ^Y^^i^ yields a cover ip : C4 — > W. By the Riemann-Hurwitz formula 
6 > 4ideg{ip) which is impossible since deg{ip) > 2. Thus ^^4,^4 is simple. 

By Corollary I3.13f a) there exist components Vi C and V2 C such that S4 contains 

'^1,3(^1 X ^2)- Let £,1 and ^3 be geometric generic points of Vi and V2, respectively, and let rj = 
Ki 3(^1, 1^3). Since ^3,^3 is simple by Lemma |4Tl there are no nontrivial homomorphisms between 
^"3 ,^3 and «^i,ii . This yields an isomorphism 

E(A'4,,)^E(A'uJ©E(A'3,i3). 

By Lemma |4.4[ there is an inclusion E{A!4^£^J ^ E(A:'4 ,j) and thus an inclusion E{A!4^£^^) ^ 
E(A'3 £3). Since A4 is simple. Lemma l43l implies that End(A4 .f^) = Z. □ 

Theorem 4.6. Suppose g > AandO < f < g. If £, is a geometric generic point of Hg, then End{.Yg £^) = Z 
and thus Xg^^ is simple. 

Proof. By Corollary 13.151 and Lemma I4.4[ it suffices to prove the result when / = 0. For / = 0, 
the proof is by induction on g with the base case g = 4 supplied by Proposition 14.51 Suppose 
g > 5 and let S be the closure of £ in Hg. By Corollary I3.14[ S intersects Ai i['Hg]'^ and there is 

an irreducible component S of S x Hg, and there are irreducible components Si of and S2 of 
'Hg_2 and S3 of Ti.^; and there are irreducible components Sr of T^g_i and Si of T^g^i) such that 
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the restriction of the clutching maps yields a commutative diagram 
(4.0.1) Si X S2 X S3 ^ Si X Sr 

Sl X S3 -SnAi,i[?^^]. 

Let rii be the generic point of S, for 1 < i < 3; similarly, let rji be the generic point of Si, and tjr 
that of Sr. Let s = ki g_2,i(ni/ 12, Hs)- By Lemma l4!4l there are inclusions 

(4.0.2) EndlA"^,,) End(A'^,^^^_^(,^,,^)) 

^T^^i^g,Ks-i,i{vc^m)) ^ End(A'^,£). 

There is a canonical isomorphism of rational Tate modules y^(A'g^s) = ^('{'^i,rii) ^ ^({'^g-i,^) ^ 
^^('^1,173)- Choose coordinates on V^(A'^^s) compatible with this decomposition. On one hand, by 
the inductive hypothesis, E^(A:'g_i = Q^. On the other hand, since ^i,,,3 is a supersingular 
elliptic curve, E(^i^^3) is the quaternion algebra ramified only at p and 00, and thus E^(^i ,,3) = 
Mat2(Q£). Therefore ^{'^g,Kg_i^,{ni,xn3)) - H^g-hr^i) ® ^(^"1,^3), and EeX^g,Kg_,^,{J^cK,^3)) acts on 
Ve{Xg,s) as diag2^_2(Q^) ® Mat2(Q^). Similarly, EH'^g,Ki,,_i(Tiix^K)) acts as Mat2(Q^) diag2^_2(Q«) 
Then 

^e{Xg,l) Q M^g,Kg_i,i{nr^7i3))^'^e^^g,\g-i{mxvR)) 
so E£(Afg,^) acts on Ve{Xg,s) as diag2,,(Q£). Thus, E({Xg^^) ^ and End{Xg^^) ^ Z. □ 

Remark 4.7. For g < 3 and 1 < / < ^, it is also true that End(A:'3^^) = Z and ^3,^ is simple for £, a 

geometric generic point of Tig. More generally, for g > i and / > 1, the statement of Theorem l4.6l 
can be proved as an application of Theorem I5.2l as in IIAP081 Application 5.7]. 

5. MONODROMY 

In this section, we determine the £-adic monodromy of components of Ti.{, for odd primes £. 
The proof uses an inductive process which depends on the degeneration results from Section |3l 
For / > 1, the base case g = 2 relies on a special case of nChaOSl Prop. 4.4]. When / = 0, the base 
case g = 4: relies on the results on endomorphism rings from SectionlH 

5.1. Integral monodromy. We summarize the discussion in IIAP071 Sec. 3.1] about Z/£- and Z/- 
monodromy. Let S be a connected fc-scheme on which the prime £ is invertible. Let n : C ^ S 
be a relative curve of compact type whose fibres have genus g. Then R^7t^{pii), or equivalently 
Pic°(C) [£], is an etale sheaf of Z /^-modules. If s is a geometric point of S, then R^7r*(^£) is equiv- 
alent to a linear representation 

Let M(?(C S,s) be the image of Pc^s,z/e and let M^(C S) be the isomorphism class of this 
image as an abstract group. If the family C — > S is clear from context, these will be denoted 
M£(S,s) and M^(S), respectively. There is a canonical polarization on Pic^(C), and thus (after 
a choice of £*^ root of unity on S) there is a symplectic pairing on Pic° ( C ) [£] . Therefore, there 
is an inclusion of groups Me{S) C Sp2^(Z/^). Similarly, for each natural number n there is a 
representation 

Pc^s,z/e" ■ 7ri(S,s) Aut{{R^7t^{fign))s). 
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Let Mzf{C S,s) = lim^ Pc^s,z/£"{'^'i-{S,s)), and let Mq^(C S,s) be the Zariski closure of 

Mz,(C ^ S,s) mAut{lim^{R^7t4^lfn))s Qe) ^ GL2^,q,. 

If C ^ S is a stable curve such that C has compact type over an open dense subscheme U C S, 
then for each coefficient ring F E {Z/£,Z£,Q('} one can define Mr(C S,s) = Ma{C\u U,s). 

One can employ an analogous formalism to define the monodromy group of a relative curve 
over a stack IINoo04[ . 

5.2. Monodromy of the hyperelliptic p-rank strata. In this section, let p and £ be distinct odd 

primes. We find the integral monodromy of the p-rank strata Tig when 1 < f < g. The integral 
monodromy of Tig, which is the same as the case f = g, already appears in |AP07l Thm. 3.4] (see 
also unpublished work of J.-K. Yu, and [Hal06, Thm. 5.1]). 

The following argument shows that to determine the monodromy of families of hyperelliptic 
curves, one may work with either Hg or Hg. 

Lemma 5.1. Let p and i be distinct odd primes and suppose g > 2. Let S C Hg be irreducible and let S be 
an irreducible component of S Xj^^^ Hg. Then M£(S) = Sp2^(Z/£) if and only ifM({S) = Sp2^(Z/£). 

Proof Since S ^ S is finite, M^(S) is a subgroup of M^(S). If M^(S) = Sp2^(Z/£) then M^(S) is 
maximal, and thus so are M^(S) and M£{S). 

Conversely, suppose M({S) = Sp2g(Z/£). Since cOg : Hg Hg is etale with Galois group 

Sym(2^ + 2), the cover S ^ S is Galois with Galois group G C Sym(2g + 2). To show M^(S) = 
Sp2^(Z/£), it suffices by the argument of |AP07l Lemma 3.3] to show that G and Sp2^{'^/^) have 
no common nontrivial quotient. This holds since the smallest integer N for which there exists an 
embedding of the finite simple group FSp^^iZ/i) into Sym(N) is N = {i^S - - i) > 2^ + 2 

IIGre03l Thm. 3]. ' □ 

Theorem 5.2. Let p and £ be distinct odd primes. Suppose g > 1 and 1 < f < g. Let S be an irreducible 



component ofHg, the p-rank f stratum in Hg. Then M^(S) = Sp2^(Z/^) and Mif{S) = Sp2^( 

Proof. The proof is by induction on g. The base cases involve the monodromy of H2 and H\, which 
follow from IICha05l Prop. 4.4]; see l, AP08. Thm. 4.5]. 

Now suppose g > 3 and 1 As an inductive hypothesis assume, for all pairs {g',f') 

where ^ < f < g' < g, that Mi{S') = Sp2^,(Z/£) for every irreducible component S' of H^^,. 

Let S be an irreducible component of Hg. Recall the degeneration types identified immediately 
before Corollary l3l4l If / = ^, let [fijijo,) = {l,g - 2,1) as in case (A); if / = ^ - 1, let 
ifijijs) = {0,g - 2, 1) as in case (B); and if 1 < / < g -2, let (/i,/2,/3^= (OJ,0)^as in case 
(C). By Corollary 13.141 there are irreducible components S of S X:^ Hg, Si of h{^, S2 of H^^^_2 

and S3 of h{^; and there are irreducible components Sr of H^^^l^ and Sl of H^^^-l'^; such that the 
restriction of the clutching maps yields a commutative diagram 

(5.2.1) Sl X S2 X S3 ^ Sl X Sr 



Ze 




SlxSs -Ai,i[S]. 

In all cases /i + /2 and /2 + fs are positive, and so Si and Sr have monodromy Sp2(g-i) (^/^)/ by 
induction and Lemma ISTT] 
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The rest of the proof is identical to that of IAP071 Thm. 3.4]. Briefly, one calculates the mon- 
odromy group of S at a point s in the image of Si x S2 x S3 under Ki^g-2,i- On one hand, there 
is an a priori inclusion M/'(S,s) C Sp2^(Z/^). On the other hand, the previous paragraph shows 
that M^(S,s) contains two distinct subgroups isomorphic to Sp2(^_i)(Z/^). A group-theoretic re- 
sult shows that Me{S, s) = Sp2^(Z/£). The result then follows from Lemma ISm 

The proof that Mz, (S) ^ Sp2g{^e) is identical. □ 

5.2.1. A p-adic complement. In this section we determine the p-adic monodromy of components of 

the p-rank strata Hg when / > 1. 

Let S be a connected scheme of characteristic p with geometric point s, and let X ^ S be an 
abelian scheme with constant p-rank /. The group scheme X[p] and p-divisible group X[p°°] admit 
largest etale quotients, X[p]'^' and X[p°°]'^*. These are respectively classified by homomorphisms 

7ti{S,s) Aut(X[p]'^*)s ^ GL/(Z/p) and 7ti{S,s) Aut(X[p~]'^*)s = GL/(Zp), whose images 
are denoted Mp(X S) and Mz^(X S), or simply Mp(S) and Mzp(S)- 

Lemma 5.3. Suppose > 1 and 1 < f < g. Let S be an irreducible component ofHg, and let S he an 
irreducible component of S x^^ Hg. Then Mp(S) ^ GL^(Z/p) if and only ifMp{S) ^ GL|(Z/p). 

Proof Since S ^ S is finite, Mp(S) is a subgroup of Mp(S). If Mp(S) ^ GL^(Z/p) then Mp(S) is 
maximal, and thus so are Mp(S) and Mp(S). 

Conversely, suppose Mp(S) = GL/(Z/p). Let S* be the closure of S inTlg - Ao[W^] and let S* 
be the closure of S in Tig — AQ[Hg]. Let T = S* — (S*)^ be the locus with p-rank smaller than /. 
Then T is nonempty by Corollary 13.151 On one hand, S* — > S* is etale since p is odd and the cover 
S* — > S* is tantamount to a partial level-two structure. On the other hand, the GL|(Z/p)-cover 
Hf := Homs((Z/p), Jac(C^^s)[p]'^') ^ S is ramified along T. Therefore, the covers Hf ^ S and 
S ^ S are disjoint, and Mp(S) = Mp(S) ^ GLf{Z/p). □ 

Proposition 5.4. Suppose g > 2 and 1 < f < g. Let S be an irreducible component of Hg. Then 
Mp(S) ^ GL^(Z/p) and Mzp(S) ^ GL/(Zp). 

Proof. First suppose f = g- When g = 2, the result for 0.2, or equivalently TWj' ^ special case of 
|Eke91l Thm. 2.1]. For g > 3, suppose as an inductive hypothesis that Mp{Hg_^) = GL^_i(Z/p). 
ByLemma|531 Mp(7Y^:J) = GL^_i(Z/p). 

Recall the diagram ( 12.4.71 ), and consider a geometric point s G Hg in the image of H\ x H.^g_2 ^ 

Til under ki^^_2,i. By the inductive hypothesis and Lemma |531 Mp{7if,s) contains two distinct 
copies of GLg_i (Z/p) and thus equals GLg(Z/p) by the argument of IIAP071 Lemma 3.2]. 

Now suppose 1 </<g— 1. By Corollary 13. 13f a) . there are irreducible components Vi C Hi 

and V2 C such that S contains ki g_i (Vi x V2). By the inductive hypothesis and Lemma 1531 

Mp{V2) = GL/(Z/p), and thus Mp(S) ^ GL/(Z/p) as well. 

The proof that M^^ (S) ^ GL/(Zp) is identical. □ 

5.3. p-rank zero: monodromy. In this section, we determine the integral monodromy of compo- 
nents of Hg under a few mild hypotheses. The monodromy group of H2 is small, since super- 
singular families of abelian varieties have finite £-adic monodromy groups; and the methods of 
IICha05| do not apply to H^ for g > 3, because the hyperelliptic Torelli locus is not a Hecke-stable 
subset of Ag. Thus our proof requires another base case when / = 0. For lack of a strategy to 
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calculate the ^-adic monodromy of Tiy we analyze the case when ^ = 4 and / = using re- 
sults on endomorphism rings from Section IH We thus determine the mod-£ monodromy group 
of components of when g > 4, for all but finitely many £. Note that in Theorem I5.7[ the set 
of exceptional primes depends on the characteristic p of the base field, but not on g, so that our 
results are valid for £ »p 0. 

Lemma 5.5. Let S he an irreducible component ofH^. 

(a) Either Mq^(S) = SpgQ^ /or all I ^ ip, or there exists a totally real field L such that Mq^{S) = 
(Rl/qSL2) xQ^ /or p. 

(b) Letse Shea geometric point. For i in a set of positive density, there exists a torus Tg c Mq^,(S, s) 
which acts irreducihly on V£{JYs). 

Proof. Let rj be the generic point of S, and consider the dichotomy of Lemma l^TI If End(A:'3^^) = Z, 
then one knows (e.g., ISerOOl Thm. 3]) that Mq^ {S) ^ Sp2g for all £ / p. 

Otherwise, if £(^"3 = L, a totally real cubic field, then S coincides with (a component of) the p- 
rank zero locus of a Hilbert modular threefold attached to L. Therefore, M(q^ (S) = R£/(qSL2 xqQ^ 
for all £ 7^ p IIYu09l Lemma 6.5]. This proves (a). 

For (b), if Mq)^(S) is the symplectic group, for each prime £ choose a CM field K{£) of degree 6 
which is inert at £. 

If instead E(A:'3 is isomorphic to a totally real cubic field L, then for £ in a set of positive 
density, £ remains inert in L. (If L is Galois over Q, this is clear from the Chebotarev theorem. 
Otherwise, the Galois closure L of L has Gal(L/Q) = Sym(3); for £ in a set of positive density, 
£ splits into two primes in L, and thus £ is inert in L.) For each such £, let fl^ G Q be a positive 
number such that —Uf is not a square mod£; then K{£) := L[^y—ae\ is a totally imaginary field 
which is inert at £. 

Then the norm one torus Tg := {R^fyqGm)^^^ xq is a suitable torus. □ 

Proposition 5.6. Let S he an irreducible component ofTil- For each £ ^ p, Mq^,(S) = Spg q^. 

Frooj. Fix a geometric point s G S. By |LP95l Thm. 3.3], there exists a connected reductive 
group G/Q and an 8-dimensional representation V of G such that for £ ^ 0, the representation 
Mq^,(S,s) Aut(y^(A'4^s)) is isomorphic to the representation G Xq Aut(y (^q Q^). From 

Zarhin's theorem and the classification of semisimple Lie algebras (see, e.g., INooOOl Lemma 1.3]), 
either G = Spg, or the representation is of Mumford type [Mum69]; and in each case, Mq^ (S) is in 
fact isomorphic to G x q for all £ ^ p.\i the representation is of Mumford type, then G is isoge- 
nous to a twist of R^/Q SL2 for some totally real cubic field K, and in particular has dimension 
nine. Therefore, to prove the claim, it suffices to show that dimQ^ Mq^ (S,s) > 10. 

By Corollary 13.141 S intersects Ai i(7^i 1). As in the proof of Theorem I5.2[ one can compute 
Mq^ (S, s) at a point s in Ai4(S). Then there are components Sl and Sr of 'Hg, and components Si 
and S3 of li!^, such that Mq)^ (S,s) contains distinct subgroups Mq^ (Sl) x Mq^,(S3) and Mq^,(Si) x 
Mq^(Sr). Moreover, by Lemma l53t a), each of M(Q^ (Sl) and Mq^ (Sr) has dimension either 21 or 
9. Therefore, dim^^ Mq^(S,s) > 10, and IV1q^(S,s) ^ SpgQ^ for all £^p. □ 

Theorem 5.7. If £ »p 0, if g > 4 and if S is an irreducible component of Tig, then (S) = Sp2g(Z/£) 
flndMz,(S) ^Sp2g(Z^). 

Proof. If ^ = 4 and S is an irreducible component of Hg, then Proposition |5.6l provides the hypoth- 
esis needed to prove M^^ (S) = Spg(Z^) for all but finitely many £, e.g., llSerOOl 8.2]. This yields the 
result for g = 4: since Ti^ has only finitely many irreducible components. For ^ > 4, the proof is 
identical to that of Theorem l5.2[ with Corollary 13. 141 being used to degenerate to a component of 
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Remark 5.8. The assertion of Theorem 15 .71 is false for TL^ and if ^ > 5. Indeed, a hyperelliptic 
curve of genus g < 2 and p-rank is supersingular. Since a supersingular p-divisible group over 
a scheme S becomes trivial after a finite pullback S ^ S, the monodromy group 1^12^ (7^°) is finite 
for g < 2. The ambiguity in Lemma 141] propagates to Lemma l53l and we do not know whether 
the assertion of Theorem l5.7l is true for "Hg. 

5.4. Arithmetic applications. The results of the previous section about the monodromy of com- 
ponents of Ti.{, have arithmetic applications involving hyperelliptic curves over finite fields. For 
example, they imply that there exist hyperelliptic curves of given genus and p-rank with abso- 
lutely simple Jacobian (Application 15 .9 D . Moreover, they give estimates for the proportion of hy- 
perelliptic curves with a given genus and p-rank which have a rational point of order £ on the 
Jacobian (Application I5.11D or for which the numerator of the zeta function has large splitting 
field (Application 15. 13|) . 

Throughout this section, F denotes a finite extension of Fp. 

5.4.1. Technical context. We do not include proofs in this section, since they are very similar to 
those found in [AP08, Section 5]. Here is a brief description of the main ideas involved. One first 
defines Tig over the category of Fp-schemes and defines the arithmetic monodromy group of a 
substack of Tig. For a relative curve tt : C S/F of genus g > 2 defined over a finite field, one 
shows that if Mf """(S) ^ Sp2^(Z/^), then Mg°"'(S) = Sp2^(Zf ); and Mg*(S) has finite index in 
GSp2,(Z,); and Mg;°'"(S) ^ Sp2^,Q^, lAPOSl Lemma 5.1]. 

Secondly, in order to use Chebotarev arguments for curves over finite fields, it is necessary 
to add rigidifying data, such as the data of a tricanonical structure, so that the corresponding 
moduli problems are representable by schemes. Recall that ^'q^ is very ample, that (O^^g) is a 
locally free -module of rank 5g — 5, and that sections of this bundle define a closed embedding 
C ^ Pg^ ^. A tricanonical {3K) structure on tt : C ^ S is a choice of isomorphism (9®^^ ^ = 
tTh, (O^^g), and the only automorphisms of a hyperelliptic curve with 3K-structure are the identity 

and the hyperelliptic involution. The moduli space 'Hg,3K of smooth hyperelliptic curves of genus 
g equipped with a 3X-structure is representable by a scheme liKS99, 10.6.5], I MFK94 , Prop. 5.1]. 

Third, since Hg may be constructed as the quotient of Hg^sK by GLs^-s, the forgetful functor 
ipg : Ti-g^^K 'Hg is open IMFK941 p. 6] and a fibration with connected fibers IINoo04l Thm. 
A.12]. Thus, if S C Hg is a connected substack and S^k = S x-^^ T^gM' then M({So,k) — Mf(S) 
|AF08l Lemma 5.2]. Since the data of a tricanonical structure exists Zariski-locally on the base, 
one can relate point counts on 'H.g^j,]^{¥) to those on 'Hg(F). Specifically, if s G 'Hg(F) is such that 
Aui{Cg,s) = {±1}, thenthefiber of 7^g,3K(F) over s consists of |GL2^(F)|/2 points 1^5991 10.6.8]. 

5.4.2. Application to simple Jacobians. Using the Q^-n^onodromy of Hg, we deduce that there exist 
hyperelliptic curves of genus g and p-rank / with absolutely simple Jacobian. 

Application 5.9. Suppose g > 1 and < f < g with f ^ if g < 2. Let S be an irreducible component 
ofHg. Then there exists s S S(F) such that the Jacobian ofCg^s is absolutely simple. 

Proof. If / > 1 or ^ > 4, then this follows from Theorems |52l and EZ] using |CO01l Prop. 4]. If 
/ = and ^ = 3, then this follows from Lemma l5!5l using flCOOll Rem. 5(i)]. □ 

Remark 5.10. For arbitrary g, it is unknown how to deduce Application 15.91 from Theorem 14.61 
On an unrelated note, under the hypotheses of Application 15.91 one can deduce that Aut(Cg,,j) = 
{ ± 1 }, which yields a new proof of IIAGP081 Thm. 3.7] . 
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5.4.3. Application to class groups. Recall that if s € Hg{¥), then Fic^{Cg^s)(¥] is isomorphic to the 
class group of the function field F(Cg^s)- The size of the class group is divisible by £ exactly when 
there is a point of order £ on the Jacobian. Roughly speaking. Application 15.111 shows that among 
all curves over F of specified genus and p-rank, slightly more than l/£ of them have an F-rational 
point of order £ on their Jacobian. 

Application 5.11. Suppose £ and p are distinct odd primes, g > and 1 < f < g. Suppose S is an 
irreducible component ofHg such that S(F) ^ ^. Let m he the image of \¥\ in (Z/£)^. There exists a 
rational function ocg^mij) £ Q(T) and a constant B = B{p,g,£) such that 



(5.4.1) 



#{sgS(F):£|#Pic° (C,,.)(F)} 



B 



#S(F) 

Iff = and g > 4:, the same result is true for all £ »p 0. 

Proof. The proof is very similar to that of BAP081 Application 5.9] and uses Theorems 15.21/15^ and 
llKS99t Thm. 9.7.13]. □ 

Remark 5.12. For £ odd, one knows that a^,i (£) = + O {1 / £^), while ag,m{£) = j^ + 0{l/£^) 
if HI 7^ 1. A formula for is given in ||Ach06L 

5.4.4. Application to zeta functions. If C/F is a smooth projective curve of genus g, its zeta function 
has the form Lc/f(T) /(I — T) (1 — qT), where Lc/]p(T) G Z[T] is a polynomial of degree 2g. The 
principal polarization on the Jacobian of C forces a symmetry among the roots of Lqi^{T); the 
largest possible Galois group for the splitting field over Q of Lqi^{T) is the Weyl group of ?>Y>2g 
which is a group of size g\2^ . 

Application 5.13. Suppose g > 1 and 1 < f < g, or that g > 4 and / = 0. Suppose p > 2g + 1 and 

that S is an irreducible component ofHg such that S(F) 7^ 0. There exists a constant y = y^g) > and 
a constant E = E{p,g) such that 

#{s e S(F) : L(2 is reducible, or has splitting field with degree < 2Sg\} 

(5.4.2) '-^ < Eq-y. 

Proof. The proof is very similar to that of IIAP081 Application 5.11] and uses Theorems 15.21/15^ and 
IIK0WO6I Thm. 6.1 and Remark 3.2.(4)]. □ 
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